SegFormer: A Topic Segmentation Model with Controllable Range of Attention

Topic segmentation aims to reveal the latent structure of a document and divide it into multiple parts. However, current neural solutions are limited in context modeling sentences feature representation candidate boundaries. This causes model suffer from inefficient sentence encoding noise information interference. In this paper, we design new text SegFormer with unidirectional attention blocks better representations. To alleviate problem interference, uses novel additional aggregator topic classification loss guide aggregate within appropriate range. addition, applies an iterative prediction algorithm search for optimal boundaries progressively. We evaluate SegFormer's generalization ability, multilingual application ability on challenging real-world datasets. Experiments show that our significantly improves performance by 7.5% benchmark WIKI-SECTION compared several strong baselines. The dataset separate normal advertisement segments product marketing essays also achieves superior evaluation other cutting-edge models.